The present invention relates generally to a still image producing method and a still image capture system. More specifically, the invention relates to a still image producing method and a still image capture system, which can capture a plurality of frames images, which are slightly shifted from each other, by utilizing unavoidable vibration, such as the movement of the hands, which usually occurs on ordinary capture conditions, to synthesize the images by a predetermined method to capture a still image of a frame having a high resolution and a low noise, when the still image is captured by means of a sensor, such as a CCD.
With the rapid development of information processing systems, such as personal computers and PDAs (personal digital assistant), and digital communication networks for connecting the information processing systems, the demands for improved techniques for capturing a high-definition image information have been increased. As systems capable of capturing a still image as digital data, digital cameras have come into wide use recently. Therefore, as an example of a technique for capturing still images, a digital camera will be described below.
FIG. 6 is a schematic block diagram of a conventional digital camera. An image to be captured is imaged on an image sensor by means of an optical system. As the optical system, a lens and a stop are used. In addition, as the image sensor, a CCD or a CMOS is used. The image sensor converts an inputted optical information to an electric information, which is outputted as a time series signal. After carrying out the sampling and gain control of the electric information outputted from the sensor, the electric information is converted into digital data by means of an A/D converter. Then, a signal processing for dividing the electric information into a luminance signal and a color signal is carried out by means of a signal processing circuit, and the results thereof are stored in a still image memory. The still image information thus stored is outputted to the outside via an output part, if necessary.
Currently, most of digital cameras use a CCD area sensor as the image sensor. The CCD area sensor comprises photodiodes arranged in the form of a two-dimensional matrix, and CCD analog shift registers. When the photodiodes receive light, the photodiodes produce electrons in accordance with the quantity of the received light to accumulate the electrons therein. The analog shift registers transfer the accumulated electrons to output the electrons as time series signals.
The digital camera is designed to capture a still image, and the still image information outputted from a conventional digital camera is an image information of a single frame. That is, assuming that the exposure time of a sensor is T, an electric information corresponding to the quantity of light received by the respective photodiodes of the sensor between time t=0 and t=T is outputted from the sensor only once in time series, and the above described signal processing of the outputted electric information is carried out, so that the digital camera outputs a still image information.
However, in the main current of digital camera, an optical system having a small diameter of xc2xc inch type format is adopted in order to meet the demand that the weight and size of the digital camera be reduced. In this case, the diagonal length of the image formed on the image sensor is only about 4.5 mm. On the other hand, CCD sensors having pixels of about 500xc3x97800 are adopted to meet the demand of the resolution. That is, photodiodes of about 500xc3x97800 are arranged on a CCD sensor. Consequently, the size of one photodiode is as small as 4.5 xcexcmxc3x974.5 xcexcm. Therefore, the number of electrons, which can be produced by the photodiode, is only about 6000 when the exposure time is {fraction (1/60)} seconds at an illuminance of 10 luxes, which is ordinary brightness. It is known that a random noise of nxc2xd occurs when n electrons are produced since the electron producing process is a stochastic process. That is, when n electrons are produced, the S/N ratio is n/nxc2xd=nxc2xd so that the S/N ratio decreases as the number of electrons decreases.
With the spread of digital cameras, the enhancement of resolution of digital cameras has been required. In order to take a still image of a high resolution which is equal to a silver halide photograph, the number of pixels must be further increased. However, if the size of the photodiode is reduced in order to increase the number of pixels, the number of produced electrons decreases, so that the S/N ratio further decreases. On the other hand, in order to increase the number of pixels without reducing the size of the photodiode, it is required to enlarge the optical system, so that this does not meet the demand that the weight be reduced.
Thus, in conventional digital cameras, there is a trade-off relationship between the resolution and the S/N ratio due to the random noise, so that it is not possible to take a high definition still image having a high resolution and a low noise while meeting the demand that the weight be reduced.
On the other hand, there is a method using a so-called swing CCD, as a method for improving the resolution without increasing the number of pixels of the CCD. This vibrates the whole CCD at a smaller amplitude than the pixel pitch of the CCD by utilizing a vibrating mechanism, such as a piezoelectric element, to intentionally shift the pixels from each other to picks up an image in order to improves the resolution of the image.
However, it is important for the swing CCD to accurately synchronize the image capture timing with the vibration of the CCD. Therefore, the vibrating mechanism is complicated, and the assembly and adjustment thereof are not easy. Moreover, it is not easy to accurately reproduce ultra micro vibration which is a fraction of the pixel pitch, so that it is not easy to ensure the stability of the system over a long period of time.
It is therefore an object of the present invention to eliminate the aforementioned problems and to provide an image processing method and a still image capture system, which can take a high definition still image having a high resolution and a high S/N ratio by means of light and simple mechanisms.
According to the present invention, it is possible to take a high definition still image having a high resolution and a low noise by synthesizing frame images, which are slightly shifted from each other by the xe2x80x9cincidental movement of an apparatus such as a cameraxe2x80x9d in taking a picture. It is conventionally conceived that such a xe2x80x9cmovement of a cameraxe2x80x9d is a bad and undesired factor to deteriorate images, so that the incidental movement of an apparatus such as a camera is conventionally actively removed. However, the inventor converted the conventional idea to make the present invention after analyzing, in detail, such a xe2x80x9cmovement of a cameraxe2x80x9d, which unavoidably occurs when taking a still image.
The present invention positively utilizes the xe2x80x9cmovement of a cameraxe2x80x9d unlike conventional systems, and is characterized by the improvement of the resolution and S/N ratio of a still image. Such a xe2x80x9cmovement of a cameraxe2x80x9d does not match with pixel pitch. Therefore, different points of an object are recorded on the respective pixels on each of frames sequentially shot. Then, if the respective frames are overlapped with each other so that the positions of the object on the frames are overlaid with each other, it is possible to obtain a still image having a greater number of pixels than that of each of the frames.
According to one aspect of the present invention, there is provided a still image producing method for producing a still image of an object by means of an image pickup element capable of capturing-an image as a plurality of pixels, the method comprising the steps of: taking pictures of the object while allowing the relative displacements between the object under substantially the same exposure time with each other and the image pickup element; producing a plurality of frame images being formed by a plurality of pixels so that different points of the object are recorded on the corresponding pixels of at least a predetermined number N (N is a natural number equal to or greater than 2) of frame images of the plurality of frame images; calculating relative displacement amounts between the positions of the object on the plurality of frame images; and overlapping the plurality of frame images after shifting the plurality of frame images from each other on the basis of the relative displacement amounts so that the positions of the object on the plurality of frame images are overlaid with each other, to produce a synthesized still image of the object as an image containing an amount of information, which is N times as large as that of each of the frame images.
According to another aspect of the present invention, a still image capture system comprises: an image pickup element for taking a picture of an object to capture an image as a plurality of pixels; and a signal processing system for calculating relative displacement amounts between positions of the object on a plurality of frame images, each of which is formed by a plurality of pixels shot by the image pickup element, under substantially the same exposure time with each other so that different points of the object are recorded on the corresponding pixels of a predetermined number N (N is a natural number equal to or greater than 2) of frame images of the plurality of frame images, the signal processing system overlapping the plurality of frame images after shifting the plurality of frame images from each other on the basis of the relative displacement amounts so that the positions of the object on the plurality of frame images are overlaid with each other, to produce a synthesized still image of the object as an image containing an amount of information, which is N times as large as that of each of the frame images.
With these constructions, the present invention has the following advantages.
First, according to the present invention, it is possible to capture a high definition still image having improved resolution and random noise while maintaining an optional system of a camera to be small and light.
According to the present invention, in order to obtain such a high definition still image, it is not required to use a so-called swing CCD having complicated mechanisms or an expensive CCD having a large number of pixels. Therefore, the costs can be reduced, and the mechanisms are simple and difficult to break down, so that it is possible to obtain superior reliability.
According to the present invention, it is possible to suitably adjust the balance between resolution and noise in accordance with the type of an object, the picture taking conditions, and the characteristics of an optical system and image sensor of a camera. In addition, the interpolation processing can be carried out any numbers of times as long as data are stored in a sequential still image memory. Therefore, the interpolation processing conditions can be modified using the feedback based on the image quality of the processed still image.
Moreover, according to the present invention, it is possible to capture a high definition still image by means of a simple hardware by positively utilizing the xe2x80x9cmovement of a cameraxe2x80x9d, which is conventionally conceived that it is bad and undesirable to take a still image. It is enough for this xe2x80x9cmovement of a cameraxe2x80x9d to correspond to an order of the picture pitch, so that it is not required to intentionally move the camera. For example, the vibration caused by the pushing operation of the shutter of the camera is enough. However, the present invention can similarly cope with the ordinary xe2x80x9cmovement of the handsxe2x80x9d, which is far greater than the vibration caused by the pushing operation of the shutter. Therefore, according to the present invention, it is possible to capture a high definition still image without being careful in the xe2x80x9cmovement of the handsxe2x80x9d in taking a picture. As a result, in comparison with conventional digital cameras, it is far easy to use a digital camera, and it is possible to obtain a higher image quality at low costs.
Thus, according to the present invention, it is possible to easily obtain a high definition still image, so that the invention has great industrial merits.